Skip to content

Draft: prove registry-first WoT Binding and Connectivity 1.1 stack - #19

Closed
marcschier wants to merge 20 commits into
copilot/3993-live-node-manager-lifecyclefrom
marcschier/wot-binding-xregistry
Closed

Draft: prove registry-first WoT Binding and Connectivity 1.1 stack#19
marcschier wants to merge 20 commits into
copilot/3993-live-node-manager-lifecyclefrom
marcschier/wot-binding-xregistry

Conversation

@marcschier

@marcschier marcschier commented Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Status and dependencies

Draft proof implementation.

What this proves

Lossless NodeSet2 ↔ WoT conversion

  • Adds a dependency-light, AOT-safe WoT lexical model and converters to Opc.Ua.Types.
  • Preserves unrecognized/unmapped JSON-LD values as digest-protected RFC 6901 residue entries in standard NodeSet Extensions.
  • Emits the deterministic, versioned uav:nodes projection only when readable-only reverse conversion does not reproduce the complete UANodeSet.
  • Supports WhenRequired, Always, and Never preservation modes. uav:nodeSet is emitted only for explicit byte archival or a demonstrated fallback, and completeness tests use Never.
  • Reconstructs from the complete native projection, an explicit envelope, or ordinary binding-authored TD/TM documents.
  • Includes bounded resolver contracts, structured diagnostics and canonical NodeSet comparison.

WoT source generation

  • Accepts .tm.json, .td.json, .tm.jsonld, .td.jsonld and explicitly opted-in .jsonld AdditionalFiles.
  • Converts them to an in-memory UANodeSet and uses the existing direct NodeSet code-generation pipeline.
  • Adds stable MODELGEN030–MODELGEN034 diagnostics for parsing, conversion and virtual-path collisions.

Live NodeManager shadow reload

  • Extends [Server] Add live NodeManager lifecycle support OPCFoundation/UA-.NETStandard#4015 with ShadowReloadAsync.
  • Adds ImmediateReloadAsync as the alternative policy: it atomically routes new requests to the replacement, invalidates affected non-durable monitored items with BadNodeIdUnknown, detaches disposed owner graphs, and rejects durable subscriptions before commit.
  • Routes new service requests to the replacement generation while existing monitored items remain on the retired generation.
  • Disposes retired generations after subscriptions drain.
  • Keeps the existing fail-closed ReloadAsync/RemoveAsync behavior unchanged.

Registry-first WoT Connectivity 1.1

  • Compiles the pinned xRegistry model and the single combined WoT-Con 1.1 NodeSet. The combined model contains every published 1.02 node plus the additive registry model in the same namespace, so the old ModelDesign is not generated a second time.
  • Adds a stable, injectable TD/TM registry with in-memory and atomic file-backed stores, versions, epochs, desired/active state, persisted labels and bounds.
  • Exposes browseable xRegistry/WoT-Con 1.1 group/resource/file nodes, FileTransfer, CRUD, label, validation, enable/default-version and detailed Refresh methods.
  • Builds TM/TD dependency closures and materializes them through runtime NodeSet lifecycle Add/ShadowReload/Remove.
  • Resolves document dependencies only from the registry snapshot (or an injected federation layer); arbitrary external document URLs are not fetched.
  • Exposes WotProjectionRetirementPolicy so deployments select graceful or immediate replacement.
  • Retains invalid versions and the previous active projection, emits resource/refresh events, and reuses lifecycle model-change notifications.
  • Adapts the existing v1.02 CreateAsset/WoTFile/CloseAndUpdate workflow to the registry without changing published signatures or clients.

Replaceable protocol binding code-behind

Adds a dependency-light planner/binder project for:

  • HTTP
  • CoAP
  • MQTT
  • Modbus TCP
  • BACnet
  • PROFINET
  • LoRaWAN
  • OPC UA

Concrete optional executor projects cover HTTP, MQTT, Modbus TCP and OPC UA. The OPC UA executor supports Read, Write, native data-change observation, Method invocation and Event subscription, including portable nsu= NodeIds.

BACnet, PROFINET, LoRaWAN and CoAP are planner/validation-only in this proof and report their non-executable capability explicitly.

Review remediation

The follow-up review fixes are included:

  • shadow-retired data monitored items remain modifiable, transferable and deletable through their owning NodeManager and trigger prompt retired-generation cleanup;
  • mqtts enables TLS, defaults to 8883, resolves credentials/trust and fails closed;
  • Modbus validates the full 16-bit address range and maps function-only forms exactly;
  • HTTP redirects are explicit, bounded and cannot forward credentials across origins;
  • registry mutations become visible only after an atomic durable store commit;
  • all generated typed WoT event fields are populated;
  • uav:nodeSet.sha256 is mandatory and malformed XML becomes structured diagnostics;
  • one resolver context enforces aggregate limits across the full conversion;
  • event affordances carry uav:eventType, persisted identities use portable nsu= ExpandedNodeIds, and HasComponent subtypes round-trip exactly;
  • generated contexts bind ua and deterministic nsN prefixes; ReferenceType model names are emitted directly in link rel, with uav:refId as definitive identifier, and name/NodeId conflicts are rejected;
  • compact mapToTypeName / congruentTypeName hints remain paired with definitive identifiers, while arbitrary instance targets remain ExpandedNodeIds;
  • the non-published uav:typedReference / refTypeName draft surface is removed;
  • readable BrowseNames/paths prefer @context prefixes (with URI-qualified QualifiedNames also accepted), authored IDs are retained during synthesis, and the proof omits uav:nodes when the semantic mapping alone is equivalent;
  • native conversion covers root/model/alias/node fields, all eight NodeClasses, values, DataType definitions, permissions, translations and XML extensions without an envelope;
  • unknown JSON-LD members survive repeated WoT ↔ NodeSet cycles through bounded residue with stable context/link/affordance selectors;
  • observable / observeproperty is documented as advertised binding support, not a restriction on core UA monitorability.
  • registry FileType write-open/write/write-position/commit paths enforce the management SecureChannel policy while read-only access remains deployment-selectable;
  • committed reload cleanup failures carry the authoritative replacement registration so the coordinator cannot retain a stale handle, and committed model/namespace notifications are still attempted;
  • immediate retirement is serialized with monitored-item and durable-subscription mutations, emits one terminal status for Reporting/Sampling data items, clears event/aggregate queues, suppresses resend, and releases old owner graphs.

Compatibility

  • Existing OPC 10100-1 v1.02 model/NodeIds/method signatures remain unchanged.
  • Existing IWotAssetProviderFactory integrations remain supported.
  • Concrete transport dependencies are isolated from Opc.Ua.WotCon.Server.
  • Planner/core projects retain the repository's legacy target-framework matrix; concrete executors target modern .NET.

Commit structure

  1. Add shadow NodeManager reload support
  2. Add lossless WoT conversion and source generation
  3. Add injectable WoT protocol bindings
  4. Add registry-first WoT Connectivity runtime
  5. Align WoT Connectivity proof to combined 1.1 spec model
  6. Align WoT converter with portable Binding identity
  7. Make WoT NodeSet conversion native-first
  8. Address WoT model concept review feedback
  9. Address WoT mapping review feedback
  10. Prefer context prefixes for WoT QualifiedNames
  11. Address registry resolution and retirement feedback

Validation

  • full Opc.Ua.Types.Tests net10.0 and net48: passed
  • source-generation suite, including all WoT AdditionalFile cases: passed
  • full Opc.Ua.WotCon.Tests and Opc.Ua.WotCon.Binding.Tests net10.0: passed
  • lifecycle-focused net10.0 and net48 suites: passed
  • registry transaction and typed-event integration suites: passed
  • full dotnet build UA.slnx -c Release: passed
  • NativeAOT win-x64 publish/run: 115 passed
  • full converter/source-generator/legacy-TFM validation completed before the final rebase
  • pinned xRegistry/WoT-Con 1.1 models match the specification repository
  • immediate/graceful lifecycle, registry FileType security and committed-reload recovery tests: passed
  • WoT materialization policy tests: 22 passed on net10.0 and net48
  • affected Opc.Ua.Server / Opc.Ua.WotCon.Server builds on net10.0 and net472: passed with 0 warnings
  • git diff --check: clean

@marcschier marcschier mentioned this pull request Jul 21, 2026
Merged
8 tasks
marcschier and others added 10 commits July 21, 2026 23:11
@marcschier
Fix WoT converter: mandatory digest, shared resolution context, resol…
8d9d084 
…ver limits

- Make uav:nodeSet.sha256 mandatory: a missing, malformed, or
  mismatched digest now produces a structured InvalidDigest/
  DigestMismatch diagnostic and no NodeSet is returned, instead of
  silently skipping integrity verification when the field was absent.
- Catch XML/deserialization failures from UANodeSet.Read and report
  them as a WotDiagnosticCode.MalformedNodeSet diagnostic instead of
  letting the exception escape the converter.
- Create exactly one WotResolutionContext per top-level conversion in
  WotNodeSetConverter.ToNodeSetCore, seeded from the converter options
  (depth, max documents, cumulative bytes), and thread that single
  context through link/thing resolution. Removes the previous
  per-link context ??= new WotResolutionContext() fallback in
  TryResolveTargetNodeId, which reset cycle/limit tracking for every
  link instead of applying bounds across the whole conversion.
- Seed WotNodeSetDocumentConverter's resolution context from its
  configured options instead of resolver defaults.
- Add WotNodeSetConverterOptions.MaxResolverDocuments/
  MaxResolverDocumentBytes/MaxResolverTotalBytes (aligned with
  WotResolverOptions defaults) plus ToResolverOptions() and
  validation, so aggregate resolver limits are configurable through
  the same options object used for the rest of the conversion.
- Add tests: missing/malformed digest, digest mismatch, malformed
  NodeSet XML without throwing, unsupported vs. accepted (base64)
  encoding, multi-link aggregate document/byte limits sharing one
  context, and sibling-link cycle detection that doesn't block
  unrelated links.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcschier
Route ownership-sensitive monitored-item ops to owning NodeManager
e8c5fc4 
After a ShadowReload the retired generation is removed from the routing
table but still owns the monitored items created on it. Modify, Delete,
SetMonitoringMode and Transfer previously iterated only the visible
routing-table managers, so a same-namespace replacement generation would
claim (and fail on) the retired items - returning BadMonitoredItemIdInvalid
and leaving the retired generation's items undrained and its address space
never disposed until another lifecycle operation or shutdown.

Dispatch each data monitored item to its recorded owning NodeManager
(grouped by owner, each owner offered only its own indices) so a retired
generation services its own items and a same-namespace replacement cannot
cross-claim them. Event/all-event handling and result indexing are
unchanged.

Trigger prompt retired-generation cleanup when the last item drains: the
master notifies the lifecycle from the Delete path when an owner is no
longer registered, and the lifecycle runs cleanup on a background task
(request execution context suppressed, coordinated via the lifecycle
semaphore) so nothing is disposed inside the request callback. Existing
fail-closed Reload/Remove behaviour and the deferred-cleanup fallback are
preserved.

Add tests proving that after ShadowReload old data items can be modified,
disabled/re-enabled, transferred and deleted on the retired generation,
new reads use the replacement, old notifications keep flowing, and the
retired generation is disposed after the final drain without another
lifecycle operation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcschier
Harden WoT protocol executors (MQTT TLS, Modbus validation, HTTP redi…
d1236f1 
…rects)

MQTT: mqtts now enables TLS and defaults to port 8883, resolving username /
password, TLS client certificate and trust anchors through the credential
provider. A declared security scheme fails closed when unresolved, and
username / password never downgrades to plaintext unless explicitly allowed.
plain mqtt stays explicit (port 1883).

Modbus: the planner enforces address 0-65535 and the address+quantity-1 range,
maps function-only forms exactly onto codes 1,2,3,4,5,6,15,16 (mnemonic or
numeric) and rejects op/function and entity/function mismatches. The executor
re-validates the range before the ushort/byte casts.

HTTP: the executor-owned HttpClient disables automatic redirects and applies a
bounded, origin-aware redirect policy that strips custom header/query
credentials across origins, refuses loops, non-http(s) schemes and https->http
downgrades, and caps hops. A caller-supplied client with a credential-bearing
form fails closed unless CallerClientHandlesRedirectSafety is set.

Adds planner, executor and E2E regression tests and updates the binding docs
and options. WotCredential gains runtime-only client certificate / trust
material.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcschier
Make WoT registry commits atomically durable and populate typed events
5b1a054 
Registry mutations now persist through a transactional store commit
before publishing the snapshot or raising Changed, and the generated
WoT V2 event types are fully populated from the coordinator arguments.

- IWotRegistryStore: replace the per-entity Upsert/Remove methods with a
  single atomic CommitAsync(snapshot). LoadAsync returns only the
  committed generation.
- FileWotRegistryStore: stage content-addressed version blobs durably,
  then switch a single manifest.json pointer with an atomic replace, so a
  crash never exposes a half-written generation; Load reads only the
  committed manifest.
- InMemoryWotRegistryStore: keep the committed snapshot behind an atomic
  reference switch (same failure semantics).
- WotRegistryService: commit durably before publishing m_snapshot or
  raising Changed for every CRUD/default/enabled/labels and projection
  path. On failure Current stays previous, no event is raised, a retry
  re-attempts persistence and a restart never sees partial data.
- WotRegistryNodeManager: populate every generated WoT event field
  (resource identity/kind, generation, phase/outcome, validation outcome,
  load state/failed node/reason, binding uri, refresh summary/request id)
  from WotMaterializationEventArgs via SetChildValue.
- Tests: fault-injection transaction tests (atomicity, no-event, retry,
  restart) and real EventFilter integration tests that verify the typed
  fields are delivered through the running server notifier chain.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcschier
Sync corrected WoT Connectivity V2 model
b3164da
@marcschier
Reject oversized Modbus quantities
4f48525
@marcschier
Fix HTTP test server analyzer warning
bf7eedd
@marcschier
Use explicit HTTP header search comparison
0d9a3b5
@marcschier
Align WoT Connectivity proof to combined 1.1 spec model
d91d553 
Regenerate the Opc.Ua.WotCon model from the single revised
Opc.Ua.WoTCon NodeSet2, which incorporates the published OPC 10100-1
v1.02 model (NodeIds 1..172, deprecated) plus the additive registry
nodes (64000+) in one namespace (http://opcfoundation.org/UA/WoT-Con/,
version 1.1.0), replacing the separate 1.02 ModelDesign and the
Opc.Ua.WotCon.V2 companion model.

Model / build:
- Pin Opc.Ua.WoTCon.NodeSet2.xml/.csv (+ xRegistry) and drop the
  V2-named artifacts; update Sync-WotConModels.ps1 mappings/docs.
- Opc.Ua.WotCon.csproj: stop compiling WotConnection.xml/csv; compile
  the combined NodeSet once as Opc.Ua.WotCon (name WotCon, model URI
  WoT-Con/). Legacy 1.02 sources kept under Design/ for reference only.

Server / tests / docs:
- Replace all Opc.Ua.WotCon.V2 namespace/type references with the
  combined Opc.Ua.WotCon generated types (AddOpcUaWotCon, ObjectIds,
  BrowseNames, DataTypeIds, ...); remove user-visible V2 wording and
  rename the registry config section to OpcUa:WotConRegistry:Server.
- Decide model-node ownership cleanly via WotConModelPartition: the
  registry NodeManager owns {WotCon, xRegistry} and loads the registry
  slice (>=64000); the legacy asset NodeManager loads the incorporated
  1.02 slice (<64000). Neither claims the same static node twice.
- Add CombinedModelPreservationTests proving exact 1.02 NodeIds, typed
  method state/results and the generated client API are preserved, and
  that the registry types now coexist in Opc.Ua.WotCon.

Source generator (NodeSet2 -> ModelDesign) fixes exposed by generating
the richer combined model with typed method arguments:
- Map placeholder BrowseNames "<Name>" to "Name_Placeholder".
- Reuse an existing explicit method-type declaration instead of
  synthesizing a colliding duplicate, and carry the concrete method's
  arguments onto it.
- Register the standard encodeable types (Argument) on the value decoder
  so NodeSet2 method argument lists actually decode (was silently empty).
- Disambiguate a by-ref output argument whose name matches an input.
- Skip synthesizing method types for base-namespace (inherited) methods
  and for standalone method-type declarations.

Deprecated 1.02 nodes remain usable and all 1.02 tests pass unchanged
(only mechanical Ua.DataTypeIds qualification where the new combined
DataTypeIds shadows Core).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@marcschier
Fix WoT proof test analyzer warnings
835476e
@marcschier marcschier changed the title Draft: prove registry-first WoT Binding and Connectivity stack Draft: prove registry-first WoT Binding and Connectivity 1.1 stack Jul 22, 2026
@marcschier marcschier closed this Jul 25, 2026
@marcschier
marcschier deleted the marcschier/wot-binding-xregistry branch July 25, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@marcschier